Model: allenai/OLMo-7B-0424-hf

Layer: 5. Neuron Index: 10602

gate+_in+

gate+_in-

gate-_in+

gate-_in-
Frequency: 0.84%. Frequency: 0.43%. Frequency: 47.32%. Frequency: 51.41%.
hook_post:
Max: 1.56;
Min: 0.00;
Avg: 0.18. 		hook_post:
Max: 0.00;
Min: -0.68;
Avg: -0.03. 		hook_post:
Max: 0.00;
Min: -0.48;
Avg: -0.05. 		hook_post:
Max: 0.39;
Min: 0.00;
Avg: 0.05.
hook_pre_linear:
Max: 2.52;
Min: 0.00;
Avg: 0.93. 		hook_pre_linear:
Max: 0.00;
Min: -1.82;
Avg: -0.31. 		hook_pre_linear:
Max: 1.96;
Min: 0.00;
Avg: 0.24. 		hook_pre_linear:
Max: 0.00;
Min: -1.77;
Avg: -0.24.
hook_pre:
Max: 1.09;
Min: 0.00;
Avg: 0.20. 		hook_pre:
Max: 0.98;
Min: 0.00;
Avg: 0.12. 		hook_pre:
Max: 0.00;
Min: -1.77;
Avg: -0.61. 		hook_pre:
Max: 0.00;
Min: -1.73;
Avg: -0.63.
swish:
Max: 0.82;
Min: 0.00;
Avg: 0.11. 		swish:
Max: 0.71;
Min: 0.00;
Avg: 0.07. 		swish:
Max: 0.00;
Min: -0.28;
Avg: -0.20. 		swish:
Max: 0.00;
Min: -0.28;
Avg: -0.21.

Prototypical activations for case gate+_in+

Extreme hook_post activations

Example 0

Example 1

Example 2

Example 3

Example 4

Example 5

Example 6

Example 7

Example 8

Example 9

Example 10

Example 11

Example 12

Example 13

Example 14

Example 15

Extreme hook_pre_linear activations

Example 0

Example 1

Example 2

Example 3

Example 4

Example 5

Example 6

Example 7

Example 8

Example 9

Example 10

Example 11

Example 12

Example 13

Example 14

Example 15

Extreme hook_pre activations

Example 0

Example 1

Example 2

Example 3

Example 4

Example 5

Example 6

Example 7

Example 8

Example 9

Example 10

Example 11

Example 12

Example 13

Example 14

Example 15

Prototypical activations for case gate+_in-

Extreme hook_post activations

Example 0

Example 1

Example 2

Example 3

Example 4

Example 5

Example 6

Example 7

Example 8

Example 9

Example 10

Example 11

Example 12

Example 13

Example 14

Example 15

Extreme hook_pre_linear activations

Example 0

Example 1

Example 2

Example 3

Example 4

Example 5

Example 6

Example 7

Example 8

Example 9

Example 10

Example 11

Example 12

Example 13

Example 14

Example 15

Extreme hook_pre activations

Example 0

Example 1

Example 2

Example 3

Example 4

Example 5

Example 6

Example 7

Example 8

Example 9

Example 10

Example 11

Example 12

Example 13

Example 14

Example 15

Prototypical activations for case gate-_in+

Extreme hook_post activations

Example 0

Example 1

Example 2

Example 3

Example 4

Example 5

Example 6

Example 7

Example 8

Example 9

Example 10

Example 11

Example 12

Example 13

Example 14

Example 15

Extreme hook_pre_linear activations

Example 0

Example 1

Example 2

Example 3

Example 4

Example 5

Example 6

Example 7

Example 8

Example 9

Example 10

Example 11

Example 12

Example 13

Example 14

Example 15

Extreme hook_pre activations

Example 0

Example 1

Example 2

Example 3

Example 4

Example 5

Example 6

Example 7

Example 8

Example 9

Example 10

Example 11

Example 12

Example 13

Example 14

Example 15

Extreme swish activations

Example 0

Example 1

Example 2

Example 3

Example 4

Example 5

Example 6

Example 7

Example 8

Example 9

Example 10

Example 11

Example 12

Example 13

Example 14

Example 15

Prototypical activations for case gate-_in-

Extreme hook_post activations

Example 0

Example 1

Example 2

Example 3

Example 4

Example 5

Example 6

Example 7

Example 8

Example 9

Example 10

Example 11

Example 12

Example 13

Example 14

Example 15

Extreme hook_pre_linear activations

Example 0

Example 1

Example 2

Example 3

Example 4

Example 5

Example 6

Example 7

Example 8

Example 9

Example 10

Example 11

Example 12

Example 13

Example 14

Example 15

Extreme hook_pre activations

Example 0

Example 1

Example 2

Example 3

Example 4

Example 5

Example 6

Example 7

Example 8

Example 9

Example 10

Example 11

Example 12

Example 13

Example 14

Example 15

Extreme swish activations

Example 0

Example 1

Example 2

Example 3

Example 4

Example 5

Example 6

Example 7

Example 8

Example 9

Example 10

Example 11

Example 12

Example 13

Example 14

Example 15